Mobile mapping and computer vision for generation of 3D site models

Cultural heritage documentation has had spectacular successes in the preservation and restoration of sites destroyed by natural disasters or by war. These successes were largely based on photographic documentation, that were translated into construction information using manual methods to include all types of 3-dimensional cues, be this stereo, shadows, shading or the simple use of geometric knowledge about a building and combining this with a single image. Computer vision and direct shape measurements now add the option for an automated analysis of the imagery, thereby enabling a more complete assessment of sites than ever before. We review a series of projects and efforts to develop technologies, systems and applications with site models at accuracies in the range of at least ± 10 cm. In our aerial workflow we use images from an UltraCam-D camera from Vexcel Imaging which delivers 16 bit pan-sharpend RGB-NIR images with a size of 11500 x 7500 pixels. The camera is able to deliver images at intervals down to 1 sec. Our workflow includes the following steps: a classification of all images, the aerial triangulation (AT) using area and feature based POIs, a dense matching to generate a dense DSM (digital surface model), a refined classification using the DSM, a ‘true’ orthophoto production, and the estimation of a DEM. In this part of the paper we will focus on the automatic AT and on dense matching only. 2.1 Automatic Aerial Triangulation Digital airborne cameras are able to deliver high redundant images which result in small baselines. Normally, the strips of images should have at least 80% forward overlap and a minimum of 20% side overlap (in urban areas 60% side overlap). This high redundancy one point on the ground can be seen in up to 15 images and the constraint motion of a plane help to find good starting solutions needed for a fully automated AT. Nevertheless, an accurate extraction of tie points is needed for a robust and accurate AT (Thurgood et al. 2004). Our POI extraction is based on Harris points and POIs from line intersections (Bauer et al. 2004). The POIs from line intersections which we call ‘zwickels’ are very suitable for urban areas. Zwickels are sections defined by two intersecting line segments, dividing the neighborhood around the intersection point into two sectors. After the POIs extraction in each image we calculate feature vectors in the close neighborhood. Feature vectors are used to find 1 to n correspondences between POIs in two images. Using affine invariant area based matching the number of candidates is further reduced. For all remaining candidates we iteratively apply an affine transformation to maximize the crosscorrelation score. As a result we get a list of corresponding points. In order to fulfill the nonambiguous criteria, only matches with a high distinctive score are retained. The robustness of the matching process is enhanced by processing a back-matching as well. Another restriction is enforced by the epipolar geometry. Therefore the RANSAC method is applied to the well known five point algorithm (Nister 2003). As a result we obtain inlier correspondences as well as the essential matrix. By decomposition of the essential matrix the relative orientation of the current image pair can be calculated. This step is accomplished for all consecutive image pairs. In order to get the orientation of the whole set, the scale factor for additional image pairs has to be determined. This is done using corresponding POIs available in at least three images. A block bundle adjustment refines the relative orientation of the whole set and integrates other data like GPS, DGPS, IMU or ground control information. Figure 2 shows an oriented block of 7 X 50 aerial images together with the used 3D tie points on the ground. The whole block of images was processed without any human interaction. Figure 1. In this example we use zwickels only to show their strength in urban areas. There are only two outliers within the best 25 matches before the epipolar constraint is applied. Corresponding POIs are connected by lines. Figure 2. 7 strips of about 50 images each (5 strips flown east-west and 2 north-south) denoted by small arrows are oriented to each other using about 70.000 tie points on the ground which are shown as white dots.